Apparatus and method for reproducing handwritten message by using handwriting data

ABSTRACT

A method including generating position information and time information of a handwriting inputted on a device; generating pixel values of an image capturing the handwriting; generating an image file for facilitating a sequential reproduction of the handwriting by storing the position information and the time information of the handwriting in a first field of the image file and storing the pixel values in a second field of the image file; and transmitting the generated image file to another device.

PRIORITY

This application is a Continuation Application of, and claims priorityunder 35 U.S.C. § 120 to, U.S. patent application Ser. No. 15/810,752,which was filed in the U.S. Patent and Trademark Office (USPTO) on Nov.13, 2017, which is a Continuation Application of, and claims priorityunder 35 U.S.C. § 120 to, U.S. patent application Ser. No. 15/053,597,which was filed in the USPTO on Feb. 25, 2016, issued on Nov. 14, 2017,as U.S. Pat. No. 9,818,036, which is a Continuation Application of, andclaims priority under 35 U.S.C. § 120 to, U.S. patent application Ser.No. 12/879,283, which was filed in the USPTO on Sep. 10, 2010, issued onMar. 15, 2016, as U.S. Pat. No. 9,288,302, and claims priority under 35U.S.C. § 119(a) to a Korean Patent Application Serial No.10-2009-0107220, which was filed in the Korean Intellectual PropertyOffice on Nov. 6, 2009, the entire disclosure of each of which isincorporated herein by reference.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates generally to reproduction of a handwrittenmessage and, more particularly, to an apparatus and method forsequentially reproducing a handwritten message using handwriting datacontaining position information and time information of the handwrittenmessage.

2. Description of the Related Art

With the development of various technologies including wired andwireless communication techniques, multimedia techniques, manufacturingtechniques for hardware like a Central Processing Unit (CPU) andassociated memory, power charging techniques, and so forth, functions ofmobile terminals are increasingly emphasized. For example, a mobileterminal conventionally transmits necessary data to a communicationpartner through voice communication. However, after transmission andreception of a text message become possible, desired data can betransmitted to the partner using a Short Message Service (SMS) includingsimple text data in a situation where voice communication is not allowedor simple data needs to be transmitted.

The SMS has evolved into a Long Message Service (LMS) for long text dataand a Multimedia Message Service (MMS) for multimedia files such asstill or moving images, through which users can be provided with variousbenefits. In particular, the MMS which transmits a multimedia file,unlike the SMS or the LMS which merely transmits text data, has been inincreasing demand from users due to extensibility in expression. Thefollowing describes a method for transmitting desired data by using theMMS.

Referring to FIG. 1, a user may create a conventional MMS message byinputting desired characters or attaching a desired image to ahandwriting recognition region. In an MMS input screen shown in FIG. 1,the handwriting recognition region includes three single recognitionregions, in each of which a single character is recognized at a time.Character information desired by the user is input by a finger or aninput device such as a stylus pen. By using a file attachment boxpositioned below a character input window, an image stored in a memorymay be retrieved for transmission.

However, conventional MMS transmits data including a previously storedimage. When a transmitting side desires to transmit handwriting data byusing the MMS, the handwriting data has to be converted into an imagefile by using a separate program, and then must be stored andtransmitted. Since the handwriting data is converted into a still image,information about the time when the user performed the handwriting isnot included in the still image.

In other words, when a receiving side receives an image included in theMMS message, the image looks like a single still image and thehandwriting data cannot be sequentially reproduced in the same order inwhich the handwritten data was created. As a result, the handwriting orsensibility of the transmitting side cannot be contemporaneouslydelivered in a sequential manner, thereby eliminating the feeling ofprogressive data entry.

SUMMARY OF THE INVENTION

The present invention is made to address at least the above problemsand/or disadvantages and to provide at least the advantages describedbelow.

Accordingly, an aspect of the present invention is to provide anapparatus and method for sequentially reproducing a handwritten messageto a receiving side using handwriting data including positioninformation and time information of the handwritten message.

According to an aspect of the present invention, there is provided amethod including generating position information and time information ofa handwriting inputted on a device; generating pixel values of an imagecapturing the handwriting; generating an image file for facilitating asequential reproduction of the handwriting by storing the positioninformation and the time information of the handwriting in a first fieldof the image file and storing the pixel values in a second field of theimage file; and transmitting the generated image file to another device.

According to another aspect of the present invention, there is provideda method including receiving an image file including a first fieldstoring position information and time information of a handwriting and asecond field storing pixel values of an image capturing the handwriting;and reproducing the handwriting on a display screen sequentially basedon the image file.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features and advantages of an embodiment of thepresent invention will be more apparent from the following detaileddescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 illustrates information transmission using a conventionalMultimedia Message Service (MMS);

FIG. 2 illustrates a process of generating a handwritten messageaccording to an embodiment of the present invention;

FIG. 3 is a block diagram of a mobile terminal for transmitting andreceiving a handwritten message according to an embodiment of thepresent invention;

FIG. 4 is a flowchart illustrating a process of generating a handwritingimage including handwriting data according to an embodiment of thepresent invention;

FIG. 5 is a flowchart illustrating a process of reproducing ahandwriting image including handwriting data according to an embodimentof the present invention;

FIGS. 6 through 8 are diagrams illustrating a field where handwritingdata is stored according to an embodiment of the present invention;

FIG. 9 is a structural diagram illustrating an internal structure ofpoint sequence data according to an embodiment of the present invention;and

FIG. 10 is a diagram illustrating a process of sequentially reproducinga handwritten message using handwriting data according to an embodimentof the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, preferred embodiments of the present invention aredescribed in detail with reference to the accompanying drawings. In thefollowing description, details such as components are provided, but theyare provided to assist a comprehensive understanding of the presentinvention. Accordingly, those of ordinary skill in the art willrecognize that various changes and modifications of the details can bemade without departing from the scope of the invention.

FIG. 2 is a diagram illustrating a process of generating a handwrittenmessage according to an embodiment of the present invention.

Referring to FIG. 2, a transmitting-side user generates a handwrittenmessage using a finger or an input device such as a stylus pen on amobile terminal. The mobile terminal generates handwriting dataincluding position information and time information regarding one ormore points forming the handwritten message, and also generates a stillimage using the position information of the handwriting data.

The still image includes a plurality of fields, one of which storespixel values acquired by imaging the handwritten message. These pixelvalues are defined as still image data which does not include timeinformation or position information regarding one or more points formingthe handwritten message. The handwriting data is included in one of theplurality of fields of the still image and is compressed.

In the present invention, an image where handwriting data is stored in aparticular field of the still image including the still image data isdefined as a handwriting image. The handwriting image is then compressedin a format such as Joint Photographic Coding Experts Group (JPEG) andthe compressed handwriting image is transmitted to the receiving side.

The internal structure of the mobile terminal which executes the processshown in FIG. 2 is shown in FIG. 3. FIG. 3 shows internal components ofthe mobile terminal for transmitting and receiving a handwriting imageaccording to an embodiment of the present invention. Referring to FIG.3, the mobile terminal includes an input/output unit 31, a controller33, and a radio communication unit 35. The input/output unit 31 includesa touch screen input unit 301 and a touch screen output unit 303. Thecontroller 33 includes an event processor 305, a first image processor307, a second image processor 313, an attached image processor 315, anda handwritten message processor 317. The radio communication unit 35includes a Multimedia Message Service (MMS) transmitter 309 and an MMSreceiver 311.

The input/output unit 31 receives an input of a handwritten message froma user and reproduces a handwriting image received from the controller33. More specifically, in the input/output unit 31, the touch screeninput unit 301 receives the input of the handwritten message from theuser and the touch screen output unit 303 reproduces the handwritingimage. The input/output unit 31 transmits the received handwrittenmessage to the controller 33.

The controller 33 interprets the handwritten message received from theinput/output unit 31 to generate handwriting data including positioninformation and time information, and generates a still image using theposition information of the generated handwriting data. The still imageincludes a plurality of fields, one of which stores still image dataincluding pixel values acquired by imaging the handwritten message.These pixel values simply express the handwritten message and do notinclude position information or time information regarding one or morepoints forming the handwritten message.

The controller 33 encodes the handwriting data included in a particularfield of the still image and transmits the encoded handwriting data tothe radio communication unit 35, detects handwriting data included inthe handwriting image received from the radio communication unit 35, andtransmits the detected handwriting data to the input/output unit 31.

More specifically, the event processor 305 of the controller 33interprets the handwritten message received from the input/output unit31 to generate the handwriting data including the position informationand the time information of one or more points forming the handwrittenmessage and generates the still image using the position information.The still image includes a plurality of fields, one of which storesstill image data that are pixel values acquired by imaging thehandwritten message. The still image data does not include the positioninformation and the time information of one or more points forming thehandwritten message. The event processor 305 transmits the handwritingimage including the handwriting data in the particular field of thestill image to the first image processor 307.

The handwriting data is stored in a field other than the field where thestill image data is stored. If the handwritten message has beengenerated on a particular image as a background, the still image dataincludes pixel values of the particular image where the handwrittenmessage is included.

The first image processor 307 encodes the handwritten message fortransmission.

The second image processor 313 decodes an image received from the radiocommunication unit 35. If the image does not include the handwritingdata as a result of decoding and interpretation thereof, the secondimage processor 313 determines the received image as a general attachedimage which does not include the handwritten message and transmits theimage to the attached image processor 315. If the image includes thehandwriting data, the second image processor 313 determines the image asa handwriting image and transmits the image to the handwritten messageprocessor 317.

The attached image processor 315, upon receiving the image having nohandwriting data, transmits the received image to the touch screenoutput unit 303, such that the received image can be output on thescreen. The handwritten message processor 317 processes the handwrittenmessage to allow sequential reproduction of the handwritten messageusing time information and position information of the handwriting dataincluded in the handwritten message. The handwritten message processor317 then transmits the processed handwritten message to the touch screenoutput unit 303.

The radio communication unit 35 transmits and receives various imagesincluding the handwriting image. More specifically, the MMS transmitter309 transmits images and the MMS receiver 311 receives images.

FIG. 4 is a flowchart illustrating a process of generating a handwritingimage including handwriting data according to an embodiment of thepresent invention.

Referring to FIG. 4, in step 401, a transmitting-side user generates ahandwritten message using a mobile terminal. The handwritten message isinput using a finger or an input device such as a stylus pen via touchscreen input unit 301. The transmitting-side user may generate thehandwritten message on an empty space or on a particular image as abackground image.

In step 403, the touch screen input unit 301 transmits the inputhandwritten message to the event processor 305, which then analyzes thereceived handwritten message to extract position information from thehandwritten message. In step 405, the event processor 305 extracts timeinformation from the received handwritten message to generatehandwriting data including the extracted position information and theextracted time information. The generated handwriting data includesposition information and time information regarding one or more pointsforming the handwritten message.

In step 407, the event processor 305 generates a still image using theposition information and transmits a handwriting image where thehandwriting data is included in a particular field of the still image tothe first image processor 307. The first image processor 307 encodes thehandwriting image. The handwriting data is stored in any one field otherthan the particular field which stores still image data. The particularfield may be an image header field, or the handwriting data may beincluded in a separate field generated at the rear of one or more fieldsforming the still image. In step 409, the encoded handwriting image istransmitted to the MMS transmitter 309 and then to a receiving-sidemobile terminal.

FIG. 5 is a flowchart illustrating a process of reproducing ahandwriting image including handwriting data according to an embodimentof the present invention.

Referring to FIG. 5, in step 501, a mobile terminal receives an imagefrom a transmitting-side mobile terminal through the MMS receiver 311.The MMS receiver 311 transmits the received image to the second imageprocessor 313. In step 503, the second image processor 313 decodes thereceived image to check if the received image includes handwriting datastored in one of a plurality of fields that form the image.

If the received image is a general attached image which does not includethe handwriting image, the received image is output through the attachedimage processor 315 and the touch screen output unit 303 in step 509. Ifthe received image is the handwriting image including the handwritingdata, the second image processor 313 transmits the decoded handwritingimage to the handwritten message processor 317 in step 505. In step 507,the handwritten message processor 317 processes the received handwritingimage to allow sequential reproduction of the handwritten message usingtime information and position information of the handwriting dataincluded in the handwriting image. The handwritten message processor 317then transmits the processed image to the touch screen output unit 303,which then sequentially reproduces the handwritten message.

The handwriting data is included in one of a plurality of fields forminga still image. However, if the handwriting data is stored in the fieldwhich stores still image data, the still image data may be damagedduring a storage process. Therefore, the handwriting data is stored in afield other than the field which stores the still image data, as will bedescribed in detail below.

FIGS. 6 through 8 are diagrams for describing a field where handwritingdata is stored according to an embodiment of the present invention.FIGS. 6 through 8, it is assumed that a single handwriting imageincludes a plurality of fields roughly divided into an image startfield, an image header field, an image data field, and an image endfield. FIG. 6 schematically illustrates an Exchangeable Image FileFormat (EXIF) information field where handwriting data is stored, FIG. 7illustrates in detail handwriting data stored in an image header field,and FIG. 8 illustrates in detail handwriting data stored in a fieldother than the image header field.

Referring to FIG. 6, the handwriting data is stored in the EXIFinformation field and still image data is stored in the image datafield. The EXIF information field, which may be included in the imageheader field, stores various information of the handwriting image, andis not displayed on the output screen of the mobile terminal duringreproduction of the handwritten message. The image data field containsthe still image data and includes pixel values of an image acquired byimaging the handwritten message, without time information or positioninformation.

In the present invention, a transmitting-side mobile terminal stores thehandwriting data in the EXIF information field and transmits thehandwriting data, and a receiving-side mobile terminal, upon receivingan image, checks if an EXIF information field of the received imagestores the handwriting data, and if so, sequentially reproduces thehandwritten message on the output screen, as described below.

Referring to FIG. 7, the image header field includes an applicationfield (APP 1), which includes an application header field, an EXIFinformation field, and a thumbnail image field. The application headerfield stores configuration and information of the application field. Thethumbnail image field stores a thumbnail image. The EXIF informationfield stores handwriting data. More specifically, the handwriting datais stored in a user comment field included in the EXIF informationfield. The handwriting data may include a point sequence symbol, asequence data length, and point sequence data. The point sequence symbolmeans a symbol indicating handwriting image including a particular imageor a symbol indicating handwriting image including no particular image.The point sequence data means handwriting data, and the sequence datalength means a total length of the handwriting data. Although not shownin FIG. 7, the handwriting data may also include the color and thicknessof lines used to form the handwritten message.

FIG. 8 illustrates the handwriting data stored in a field other than theimage header field. Referring to FIG. 8, a portion of the handwritingdata is stored in the image header field and the other portion thereofis stored in a separate field generated at the rear of the image endfield. While the point sequence symbol and the sequence data length arestored in the EXIF information field and the point sequence data isstored in a separately generated space in FIG. 8, this may be changedaccording to user's setting. For example, any information forming thehandwriting data is not stored in the image header field and allinformation forming the handwriting data may be stored in a separatelygenerated space.

Referring to FIG. 9, the point sequence data is time information ofcoordinates of consecutively input points, and may roughly include ahandwriting data header, decoding information, a stroke start, strokeattributes, a stroke point, and data end.

In FIG. 9, in the handwriting data header, the number of points formingthe handwritten message or a marker character forming the handwrittenmessage is recorded and the number of all points is also included. Thedecoding information includes version information and other information.The stroke start, the stroke attributes, and the stroke point can beclassified for each stroke, and their sizes may change according to atotal number of strokes. The data end means the end of the handwrittenmessage. The structure shown in FIG. 9 is merely an example, and issubject to change according to the type and size of the timeinformation.

Below is a description of the process of reproducing the handwrittenmessage by using the handwriting data based on the structure of themobile terminal.

FIG. 10 is a diagram illustrating a process of sequentially reproducingthe handwritten message by using the handwriting data according to anembodiment of the present invention.

Referring to FIG. 10, the second image processor 313 of the controller33 checks if the received image includes the handwriting data, and, ifso, sequentially reproduces the handwritten message on the touch screenoutput unit 303 using the time information and the position informationincluded in the handwriting data. For example, if a handwritten message“Be an impressive person!” is received, the message is sequentiallyreproduced on the touch screen output unit 303 by using time informationand position information generated according to coordinates of pointsforming the handwritten message. The handwriting data is stored in aparticular field of the image and is not displayed on the output screen.However, when a mobile terminal does not have a function of detectingthe handwriting data included in the image, the mobile terminal cannotcheck the handwriting data and thus displays the image on the touchscreen output unit 303.

As apparent from the foregoing description, the present inventionsequentially reproduces the handwritten message by using the handwritingdata including time information and position information regarding oneor more points forming the handwritten message, thus allowing expressionof the handwritten message carrying the sensibility of the transmittingside.

While the invention has been shown and described with reference to acertain preferred embodiment thereof, it will be understood by thoseskilled in the art that various changes in form and details may be madetherein without departing from the spirit and scope of the invention asdefined by the appended claims and equivalents thereof.

What is claimed is:
 1. A method for operating an electronic device,comprising: receiving, via a touch screen of the electronic device, aninput of a handwriting; identifying position information and timeinformation regarding one or more points forming the handwriting;identifying pixel values of an image capturing the handwriting;obtaining an image file for sequentially reproducing the input of thehandwriting, the image file comprising a header field including at leasta portion of the position information and the time information and animage data field storing the pixel values; and in response to a requestfor sequentially reproducing the input of the handwriting, reproducing,via the touch screen, the handwriting using the image file.
 2. Themethod of claim 1, further comprising transmitting the image file to anexternal electronic device for sequentially reproducing the handwritingat the external electronic device.
 3. The method of claim 1, wherein theimage file comprises an image header field, the image data field, animage start field, and an image end field, the image header fieldincluding the header field.
 4. The method of claim 1, wherein aremaining portion of the position information and the time informationis stored in a new field of the image file.
 5. The method of claim 1,wherein the position information is obtained by using coordinates of theone or more points forming the handwriting.
 6. The method of claim 1,wherein the time information includes information about times at whichthe one or more points forming the handwriting are identified.
 7. Anelectronic device, comprising: a transceiver; a touch screen; and acontroller configured to: receive, via the touch screen, an input of ahandwriting; identify position information and time informationregarding one or more points forming the handwriting; identify pixelvalues of an image capturing the handwriting; obtain an image file forsequentially reproducing the input of the handwriting, the image filecomprising a header field including at least a portion of the positioninformation and the time information and an image data field storing thepixel values; and in response to a request for sequentially reproducingthe input of the handwriting, reproduce, via the touch screen, thehandwriting using the image file.
 8. The electronic device of claim 7,wherein the controller is further configured to transmit, via thetransceiver, the image file to an external electronic device forsequentially reproducing the handwriting at the external electronicdevice.
 9. The electronic device of claim 7, wherein the image filecomprises an image header field, the image data field, an image startfield, and an image end field, the image header field including theheader field.
 10. The electronic device of claim 7, wherein a remainingportion of the position information and the time information is storedin a new field of the image file.
 11. The electronic device of claim 7,wherein the position information is obtained by using coordinates of theone or more points forming the handwriting.
 12. The electronic device ofclaim 7, wherein the time information includes information about timesat which the one or more points forming the handwriting are identified.13. An electronic device comprising: a transceiver; a display; and acontroller configured to: receive, via the transceiver, an image filerelated to a handwriting from an external electronic device, wherein theimage file is for sequentially reproducing an input of the handwritingand comprises a header field including at least a portion of positioninformation and time information regarding one or more points formingthe handwriting and an image data field storing pixel values of an imagecapturing the handwriting; identify a request for reproducing the inputof the handwriting; and in response to the request, sequentiallyreproduce, via the display, the handwriting using the image file. 14.The electronic device of claim 13, wherein the controller is furtherconfigured to extract the position information and the time informationincluded in the image file.
 15. The electronic device of claim 13,wherein the position information is obtained by using coordinates of theone or more points forming the handwriting.
 16. The electronic device ofclaim 13, wherein the time information includes information about timesat which the one or more points forming the handwriting are identified.